EpiExpert Crowd-Forecasting Performance Board
This our own evaluation of individual forecasts submitted to the EpiExpert model. This model is submitted to the German and Polish Forecast Hub each week.
These evaluations are preliminary - this means we cannot rule out any mistakes and the plots and analyses are subject to change. The evaluations are also not authorised by the German Forecast Hub team.
If you have questions or want to give feedback, please create an issue on our github repository.
Available Forecasts
Forecast Visualisation
Germany
2020-10-12
2020-10-19
2020-10-26
2020-11-02
2020-11-09
2020-11-16
2020-11-23
2020-11-30
2020-12-07
Poland
2020-10-12
2020-10-19
2020-10-26
2020-11-02
2020-11-09
2020-11-16
2020-11-23
2020-11-30
2020-12-07
Ranking by pairwise comparison
model compare_against ratio pval adj_pval theta_i theta_j1: EpiNow2 sophiemeakin NA NA NA 1 1 2: EpiNow2 Joel NA NA NA 1 1 3: EpiNow2 seb NA NA NA 1 1 4: EpiNow2 nib NA NA NA 1 1 5: EpiNow2 seabbs NA NA NA 1 1 —
221: anonymous_Cat anonymous_Cat 1 1 1 1 1 222: Sqxleaxes Sqxleaxes 1 1 1 1 1 223: hababylon hababylon 1 1 1 1 1 224: anonymous_Okapi anonymous_Okapi 1 1 1 1 1 225: anonymous_Shrew anonymous_Shrew 1 1 1 1 1 
Score Ranking
Only those forecasters are named who have submitted a forecast in the most recent evaluated week.
Overall
Overall separate
Germany
Poland
Score Overview
Overview of all the models and metrics
Overall
Overall separate
Germany
Overall
2020-10-12
2020-10-19
2020-10-26
2020-11-02
2020-11-09
2020-11-16
2020-11-23
2020-11-30
2020-12-07
Poland
Overall
2020-10-12
2020-10-19
2020-10-26
2020-11-02
2020-11-09
2020-11-16
2020-11-23
2020-11-30
2020-12-07
Overiew of Scores by Horizon
Overview of all models and metrics stratified by forecast horizons
horizon = 1
Overall
Overall separate
Germany
Poland
horizon = 2
Overall
Overall separate
Germany
Poland
horizon = 3
Overall
Overall separate
Germany
Poland
horizon = 4
Overall
Overall separate
Germany
Poland
Interval Score Components
Weighted interval score broken down into its components “overprediction penalty”, “underprediction penalty” and “sharpness”
Overall
Overall separate
Germany
Overall
2020-10-12
2020-10-19
2020-10-26
2020-11-02
2020-11-09
2020-11-16
2020-11-23
2020-11-30
2020-12-07
Poland
Overall
2020-10-12
2020-10-19
2020-10-26
2020-11-02
2020-11-09
2020-11-16
2020-11-23
2020-11-30
2020-12-07
Calibration
Interval Coverage
Overall
Overall separate
Plot of the percentage of true values captured by each prediction interval
Quantile Coverage
Overall
Overall separate
Plot of the percentage of true values below each predictive quantile level
Bias
Overall
Overall separate
Separate by date
2020-10-12
2020-10-19
2020-10-26
2020-11-02
2020-11-09
2020-11-16
2020-11-23
2020-11-30
2020-12-07
